Expand list of Docker images by marcosbento · Pull Request #66 · ecmwf/ci-infrastructure

marcosbento · 2026-09-16T15:23:37Z

Description

As per PR title -- each commit adds a new distro image.

The content of the images was extracted from the information in runners-infra and at the same time trying to mimic the already existing dockerfiles.

Contributor Declaration

By opening this pull request, I affirm the following:

All authors agree to the Contributor License Agreement.
The code follows the project's coding standards.
I have performed self-review and added comments where needed.
I have added or updated tests to verify that my changes are effective and functional.
I have run all existing tests and confirmed they pass.

Copilot

🟡 Changes recommended

New dependent images bypass pre-merge builds, and several platform notes are outdated.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Expands public CI coverage across eight additional Linux release platforms.

Changes:

Adds base, compiler, and Boost/Qt images.
Documents the expanded image matrix.
Extends manual image contract smoke tests.

File summaries

File	Description
`public-images/ubuntu22.04/base/Dockerfile`	Adds Ubuntu 22.04 base.
`public-images/ubuntu22.04/gfortran11/Dockerfile`	Adds GCC/GFortran 11.
`public-images/ubuntu22.04/gfortran11-boost-qt6/Dockerfile`	Adds Boost and Qt6.
`public-images/ubuntu26.04/base/Dockerfile`	Adds Ubuntu 26.04 base.
`public-images/ubuntu26.04/gfortran15/Dockerfile`	Adds GCC/GFortran 15.
`public-images/ubuntu26.04/gfortran15-boost-qt6/Dockerfile`	Adds Boost and Qt6.
`public-images/rocky9/base/Dockerfile`	Adds Rocky Linux 9 base.
`public-images/rocky9/gfortran11/Dockerfile`	Adds GCC/GFortran 11.
`public-images/rocky9/gfortran11-boost-qt5/Dockerfile`	Adds Boost and Qt5.
`public-images/rocky10/base/Dockerfile`	Adds Rocky Linux 10 base.
`public-images/rocky10/gfortran14/Dockerfile`	Adds GCC/GFortran 14.
`public-images/rocky10/gfortran14-boost-qt6/Dockerfile`	Adds Boost and Qt6.
`public-images/fedora43/base/Dockerfile`	Adds Fedora 43 base.
`public-images/fedora43/gfortran15/Dockerfile`	Adds GCC/GFortran 15.
`public-images/fedora43/gfortran15-boost-qt6/Dockerfile`	Adds Boost and Qt6.
`public-images/fedora44/base/Dockerfile`	Adds Fedora 44 base.
`public-images/fedora44/gfortran16/Dockerfile`	Adds GCC/GFortran 16.
`public-images/fedora44/gfortran16-boost-qt6/Dockerfile`	Adds Boost and Qt6.
`public-images/debian12/base/Dockerfile`	Adds Debian 12 base.
`public-images/debian12/gfortran12/Dockerfile`	Adds GCC/GFortran 12.
`public-images/debian12/gfortran12-boost-qt6/Dockerfile`	Adds Boost and Qt6.
`public-images/debian13/base/Dockerfile`	Adds Debian 13 base.
`public-images/debian13/gfortran14/Dockerfile`	Adds GCC/GFortran 14.
`public-images/debian13/gfortran14-boost-qt6/Dockerfile`	Adds Boost and Qt6.
`IMAGES.md`	Catalogues platforms and toolchains.
`.github/workflows/smoke-test-runners.yml`	Adds image contract checks.

Review details

Files reviewed: 26/26 changed files
Comments generated: 5
Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

mcocdawc · 2026-09-17T10:06:13Z

+          - image: ubuntu22.04-gfortran11-boost-qt6
+            declares: ubuntu22.04/gfortran11-boost-qt6
+            cxx: g++-11
+            cxx-major: '11'


the smoke-test-runners should not go through all images.

It's task is about quickly testing if the runner-groups and the S3 storage work.

Especially if people suspect connection issues for failing tests, they can just dispatch this test.

-----–––––––––––-------------------------
The testing of dependents is already adressed

mcocdawc · 2026-09-17T08:28:58Z

Hey @marcosbento,

Thank you for this great addition.

If we are so drastically expanding the number of images I would like to add a few more tests.
It's fully on me, that the coverage was so lackluster before, sorry.
At the moment only the base images are tested for PRs and I would like to also do the toolchain variants.
I'll add this just now + I'll go through the copilot review; I have the feeling it flagged false positives.

I just saw that you are in draft mode anyway. :-D

marcosbento · 2026-09-17T08:38:49Z

@mcocdawc , I did some expanding of the tests, ensuring also that "derived" images are rebuild every time the "base" image is updated. Please have a look at my latest commits.

This is just all draft, of course... Hopefully it will inspire you! :-)

mcocdawc

I incorporated tests in #67 and ported the build-image.sh to python. It's logic became too complicated to remain bash #68

Could you update with main.

mcocdawc · 2026-09-17T10:06:13Z

+          - image: ubuntu22.04-gfortran11-boost-qt6
+            declares: ubuntu22.04/gfortran11-boost-qt6
+            cxx: g++-11
+            cxx-major: '11'


the smoke-test-runners should not go through all images.

It's task is about quickly testing if the runner-groups and the S3 storage work.

Especially if people suspect connection issues for failing tests, they can just dispatch this test.

-----–––––––––––-------------------------
The testing of dependents is already adressed

marcosbento · 2026-09-17T15:01:28Z

@mcocdawc, rebased and trimmed the changes, which are now solely focused on extending the list of Dockerfiles.

mcocdawc

Looks very good. Thank you!

Before approving I have two simple comments

mcocdawc · 2026-09-17T15:37:07Z

    unpacked.mkdir()
    with tarfile.open(staging / "source.tgz") as tar:
-        tar.extractall(unpacked, filter="data")
+        tar.extractall(unpacked)


is there a reason to remove the filter? It's slightly safer to have it.

In order to align the Ubuntu 22.04 image with the existing Ubuntu 22.04 VM the Dockerfile installs Python 3.11, and unfortunately Python 3.11 doesn't support the 'filter' parameter.

I have amend the commit that does that specific one line change to document the reason behind the change.

thank you, makes sense.

Actually I read a bit on, because I was surprised that it did work before even with python 3.11.
Indeed it was backported to 3.11.4.

For the test it's not so important, but for the future (un)taring something that comes potentially from user input the option is great. And code from this repo possibly has to do that.

I would rather bump the minimum required version in the pyproject toml to 3.11.4 and keep filter="data"
Or is there a hard blocker that we cannot do that? Then I am also fine to go without the filter argument.

Copilot

🟡 Changes recommended

Three RPM-based base images need DNF conflict handling for replacing curl-minimal with curl.

Get a fresh assessment by requesting another Copilot review.

Review details

Files reviewed: 27/27 changed files
Comments generated: 3
Review effort level: Balanced

mcocdawc · 2026-09-17T15:52:45Z

btw: the fact that https://github.com/ecmwf/ci-infrastructure/actions/runs/35240521586

fails for the new images is expected. Of course they cannot be downloaded if they were not uploaded yet. (which they will be, once they are in master).

mcocdawc · 2026-09-18T15:43:15Z

Hey @marcosbento ,

I made a few important changes in #71

the base ships no gcc
make gccN part of the image name
require all compilers to support openMP. This requires mostly the clang compilers to have additional packages; the gnu compilers generally come with openMP support.

marcosbento · 2026-09-20T07:21:06Z

Hi @mcocdawc,

Rebased the branch, and amended the existing commits to align each of the images with the new structure introduced in #71. All checks are 🟢.

mcocdawc · 2026-09-20T15:33:23Z

Wonderful, the only remaining question is still the filter="data".

marcosbento · 2026-09-20T19:50:52Z

Regarding the filter data issue, here is the context for the "choices" that were made:

Ubuntu 22.04
- the distro provides only Python 3.10 or 3.11.0rc1, so there is no way to simply install 3.11.4
  - Python 3.10 cannot be used because of restriction at ci-infrastructure's pyproject.toml
  - Python 3.11.0rc1 is simply is not stable enough to be used here!
- attempts to use the DeadSnakes repo to install a Python 3.11.4+ version failed by causing issues with the system python3-pip (?!)
- the only viable solution was to build 3.11.6 from sources (which has no problem handling the filter data)
Debian 12
- the distro provides only 3.11.2 (this is the one that is not able to handle the filter data)
- to avoid building from sources, I adjusted the test_hpc_transfer to not use the filter if not supported.

TarFile.extractall(filter="data") was added in Python 3.12 and backported to 3.11.4 (PEP 706), and the HPC transfer test relies on it. Raise the floor from 3.11 to 3.11.4 in pyproject.toml, and align the interpreter gates and the documentation that state it.

marcosbento · 2026-09-21T17:22:24Z

Hi @mcocdawc

As agreed in our last chat, I have implemented the following:

updated the minimum version of Python to 3.11.4 in the .pyproject.toml et al
updated the Debian 12 image to install Python 3.11.16 (from sources, as is done in Debian 11, and Ubuntu 22.04)
discarded the changes related to not using filter="data" in the HPC transfer test

mcocdawc

Thank you very much! You can merge whenever you like.

marcosbento requested a review from mcocdawc September 16, 2026 15:23

mcocdawc requested a balanced review from Copilot September 16, 2026 15:33

Copilot started reviewing on behalf of mcocdawc September 16, 2026 15:33 View session

Copilot AI reviewed Sep 16, 2026

View reviewed changes

marcosbento force-pushed the task/expand_docker_image_list branch from de6e225 to 29b3c3b Compare September 17, 2026 06:50

mcocdawc mentioned this pull request Sep 17, 2026

Build, test inside, then push CI images #67

Merged

mcocdawc reviewed Sep 17, 2026

View reviewed changes

marcosbento force-pushed the task/expand_docker_image_list branch from ed36cb9 to 87f2b53 Compare September 17, 2026 14:33

marcosbento marked this pull request as ready for review September 17, 2026 15:04

mcocdawc requested a balanced review from Copilot September 17, 2026 15:39

Copilot started reviewing on behalf of mcocdawc September 17, 2026 15:40 View session

mcocdawc reviewed Sep 17, 2026

View reviewed changes

Copilot AI reviewed Sep 17, 2026

View reviewed changes

Comment thread public-images/fedora43/base/Dockerfile

Comment thread public-images/fedora44/base/Dockerfile

Comment thread public-images/rocky10/base/Dockerfile

mcocdawc mentioned this pull request Sep 18, 2026

images: ship OpenMP for every compiler an image advertises #70

Merged

marcosbento force-pushed the task/expand_docker_image_list branch 3 times, most recently from 358b1e5 to 8950eec Compare September 20, 2026 07:00

marcosbento force-pushed the task/expand_docker_image_list branch 3 times, most recently from ff69da3 to bcc0a6f Compare September 20, 2026 19:22

marcosbento added 2 commits September 21, 2026 17:25

Add Debian 12 CI images

00ee7ce

marcosbento added 7 commits September 21, 2026 17:31

Add Debian 13 CI images

373d7dc

Add Rocky Linux 9 CI images

3f53eb3

Add Rocky Linux 10 CI images

16307eb

Add Fedora 43 CI images

99eba0b

Add Fedora 44 CI images

29728fb

Add Ubuntu 22.04 CI images

0248647

Add Ubuntu 26.04 CI images

6a15b95

marcosbento force-pushed the task/expand_docker_image_list branch from bcc0a6f to 6a15b95 Compare September 21, 2026 16:35

mcocdawc approved these changes Sep 21, 2026

View reviewed changes

mcocdawc merged commit ce6c5a7 into main Sep 22, 2026
59 checks passed

mcocdawc deleted the task/expand_docker_image_list branch September 22, 2026 11:53

Conversation

marcosbento commented Sep 16, 2026

Description

Contributor Declaration

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

mcocdawc Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mcocdawc commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

marcosbento commented Sep 17, 2026

Uh oh!

mcocdawc left a comment

Choose a reason for hiding this comment

Uh oh!

mcocdawc Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

marcosbento commented Sep 17, 2026

Uh oh!

mcocdawc left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mcocdawc Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marcosbento Sep 20, 2026

Choose a reason for hiding this comment

Uh oh!

mcocdawc Sep 20, 2026

Choose a reason for hiding this comment

Uh oh!

mcocdawc Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mcocdawc commented Sep 17, 2026

Uh oh!

mcocdawc commented Sep 18, 2026

Uh oh!

marcosbento commented Sep 20, 2026

Uh oh!

mcocdawc commented Sep 20, 2026

Uh oh!

marcosbento commented Sep 20, 2026

Uh oh!

marcosbento commented Sep 21, 2026

Uh oh!

mcocdawc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mcocdawc commented Sep 17, 2026 •

edited

Loading

mcocdawc left a comment •

edited

Loading

mcocdawc Sep 17, 2026 •

edited

Loading

mcocdawc Sep 20, 2026 •

edited

Loading